A functional operator-based morphological analysis of Japanese

A universal set of functional operators as proposed in Role and Reference Grammars can be used to provide a robust morphology analyser development scheme, which gives the developer of the analyser a clear guiding principle guaranteeing the exhaustiveness of his grammar from the inception of the development task, freeing him from the complex bookkeeping of continuation lexicons often associated with typical nite state lexical transducers. 1 Agglutinative morphology and its diiculties An agglutinative language is deened by Whaley 13] as \ language in which strings of morphemes occurring within a word are easily segmentable." In contrast with fusional languages such as Latin and Greek, morphemes in an ag-glutinative language are easier to identify because they carry independently identiiable semantic or grammatical meanings. As a result, one expects that morphological analysis of an agglutinative language like Japanese should not present much diiculty. But, from a viewpoint of natural language processing, analysis of agglutinative morphology involves many diiculties which might go unnoticed even by researchers in NLP. Yamashita & Matsumoto 12] discuss the problem of tokenization with regard to both segmented languages like English and unsegmented languages like Japanese. In Japanese orthography, a sentence consists of a string of letters with no spaces between words. So, the identiication of words, or tokenization, is a dual process of segmenting, which identiies single-morpheme segments, and rounding up, which identiies multi-lexeme segments. In order to achieve eeciency in the process, Yamashita & Matsumoto propose a character-oriented minimal unit, which they call MF (morpho-fragment). Unlike lexemes and morphemes, MFs are strictly character-based building blocks of a language. In the case of Japanese and Chinese, every single character is taken to constitute an MF. It is this kind of new level of abstraction which is involved in morphological analysis in general. Kimmo-type two-level lexical transducers are often used to analyse Japanese morphology as in Tateno 10]. In order to cope with an indeenite number of